Windows 11 AI features confirmed for version 24H2

• Microsoft plans several new on-device AI features for Windows 11 24H2.
• Some of these features include Windows Recall, Windows Studio Effects, Live Captions, Auto SR, Voice Clarity, Voice Focus, Click to Do, Cocreator for the Paint app, and Restyle Image for the Photos app.
• This is in addition to various changes coming to Copilot for Windows 11.

Windows 11 24H2 (2024 Update) has been available since October 1 with a slew of new features for existing eligible computers, but this initial rollout is missing the AI features. The main reason for this is that Microsoft needs more time to make sure they’re ready. However, the initial wave of AI features is already known, and it all starts with the integration of the Windows Copilot Runtime.

In this new version of the operating system, you can expect several new AI features exclusive for Copilot+ PCs that will run on-device, including Windows Recall, Windows Studio Effects, Live Captions (real-time translations), Auto Super Resolution, Voice Clarity, Click to Do, Voice Focus, Cocreator for the Paint app, and Restyle Image for the Photos app.

The new features are possible thanks to the new “Windows Copilot Runtime,” a system component on Windows 11 that integrates more than 40 AI models that run locally on the computer and provides the infrastructure to maintain them.

This also allows developers to integrate their apps with AI functionalities using the new APIs through the Windows Copilot Library.

Windows 11 AI features for version 24H2

Here’s an overview of the new features powered by AI coming to Windows in 2024 and 2025.

Windows Recall

Windows Recall (or Recall) is a new feature that works as a photographic memory that runs in the background, taking snapshots of everything you do on the computer. Using several AI models operating locally on the device, the feature analyzes and makes every content searchable using natural language, whether text or image.

The Recall feature is an exclusive feature for Copilot+ PCs that includes compatible hardware, including a processor with NPU, 16GB of RAM, and 256GB of SSD.

You will have the choice to turn the feature on or off from the out-of-box experience (OOBE), but will be able to enable or disable Recall manually from the Settings app or Group Policy.

Windows Studio Effects

Windows Studio Effects (or Studio Effects) is a combination of features that uses AI to enhance video calls and audio quality using a Neural Processing Unit (NPU).

Currently, there are five main effects, including Automatic Framing, Background Blur, Eye Contact, Voice Focus, Portrait light, and Creative filters.

• Automatic Framing: Keeps you centered in the camera view, even if you move around.
• Background Blur: It helps you to blur the background behind you during calls, providing more privacy or hiding clutter. This feature includes two options, including “Standard Blur” and “Portrait Blur.”
• Eye contact: Since you might be looking at your screen during a call, this feature makes it appear that you’re maintaining eye contact with other people. This feature includes two options, “Standard” and “Teleprompter.”
• Voice Focus: This effect reduces background noise from your microphone, making your voice clearer during calls. (Windows 11 includes two variants of Voice Focus, including one that works with NPU and another that uses software only.)

It’s important to note that Studio Effects have been part of Windows 11 for some time, but starting with version 24H2, Microsoft plans to make these features more broadly available on Copilot+ PCs.

Live Captions with translation

Although Windows 11 has had the Live Captions feature for a while, it’s now being updated with live translations. Live Captions is a feature designed to help everyone, including people with hearing issues, to understand audio through automatic transcriptions.

As part of the AI integration in the operating system, the feature can also translate conversations from a video or voice-only call to your native language, including videos you watch from YouTube or other sources. Technically, the feature can translate any audio that passes through the computer in real time.

The audio can be live or pre-recorded from any app or video. The feature can translate over 40 languages into English even without an internet connection, as it requires an NPU to work.

It’s important to note that this feature no longer transcribes the audio but can also translate the audio language in real time.

Voice Clarity

On Windows 11, Voice Clarity is an AI-powered feature that removes background noise, cancels echoes, and reduces real-time reverberation on video or voice-only calls or when recording audio.

In addition, this feature will also benefit PC games with online communication. Apps can toggle “Deep Noise Suppression” for voice-only or generic audio content. Voice Clarity ensures a clear voice in online meetings and smoother online communication.

Voice Clarity works automatically on supported applications with supported hardware that includes NPU. You won’t find an option that says turn on Voice Clarity.

Click to Do

If you know about “Cicle to Search” on Android or “Google Lens” on Chrome, then you already know the function of “Click to Do.” The “Click to Do” feature uses AI to scan and make sense of the content on the screen, and then it provides a number of actions you can take.

How does it work? Microsoft will implement this feature in the “Windows key.” When you press and hold the key and click on the screen, an overlay will appear with a context menu that allows you to perform different tasks. On the top of the screen, you can use the search box to search the current that also integrates with Windows Recall.

Smarter Windows Search 

Although that’s not the official name, the Windows Search feature will now work with AI. The change will allow users to search for apps, files, and settings with natural language from File Explorer, Settings app, and Taskbar instead of relying on keywords and information from the file.

Auto Super Resolution

Auto Super Resolution (SR) is a feature that uses AI to increase game resolution and visual quality, providing a smoother, more efficient experience that scales across hardware, including better frame rates. In other words, Super Resolution uses AI to upscale games and improves frame rates and image quality.

The feature is similar to Nvidia’s DLSS, AMD’s FidelityFX Super Resolution (FSR), and Intel’s Xe Super Sampling (XeSS), which also uses AI to upscale games. 

As part of this feature, Microsoft also created the “DirectSR” API in partnership with Nvidia, AMD, and Intel to integrate SR into the next generation of games.

Microsoft will exclusively offer the feature for Copilot+ PCs with the latest Qualcomm Snapdragon X Elite processors, and it will only be available for a select number of games.

Paint with Cocreator and Generative Fill and Erase

In the Paint app, Microsoft is also adding a new Cocreator feature that scans your drawing instead of creating an AI image for you and helps you create a new artwork, which you can refine, edit, and evolve with your ideas.

This feature uses a diffusion-based algorithm to produce a high-quality image with minimum effort.

Cocreator even includes a setting to make the drawing more literal or more expressive.

Cocreator is an AI feature exclusive for Copilot+ PCs, but it shouldn’t be confused with “Image Creator,” which is also an AI feature that uses the cloud to convert a text prompt into an image.

Generative Fill

In Paint, “Fill” is a feature that allows you to select a region of an image, and then, with a text prompt, it generates new elements to “fill” that specific area. This feature will initially be available for Copilot+ PCs, but for those devices featuring the latest Qualcomm processors.

Generative Erase

In Paint, “Erase” is a feature that allows you to select and erase an area of an image using AI. (You’ve probably seen a similar feature on your phone.)

This particular feature is expected to ship to all devices, including non-AI computers.

Photos Restyle Image, Image Creator, and Super Resolution

In the Photos app, as part of Windows 11 24H2 and exclusive to the Copilot+ PCs, Microsoft is adding a new “Restyle Image” that uses AI with the device’s NPU to change the style of a photo using image generation and photo editing techniques.

Using the feature, you can select a picture, and you can then transform the image with new styles, similar to using filters on your phone. You can even use a text prompt to create a new background.

Image Creator

The Photos app also gets the same “Image Creator” feature found in the Paint app, which allows you to create AI images with a single text prompt. This feature isn’t exclusive to Copilot+ PCs because it doesn’t require an NPU since the process happens in the cloud with Copilot using the DALL-E language model.

Super Resolution

Super Resolution is a feature that will allow you to upscale images using AI. The feature will be available for Copilot+ PCs, and it’ll use the NPU to upscale and sharpen an image up to eight times from the original resolution. 

Microsoft is also promising that Super Resolution in Photos will be able to restore those old low-quality images.

Copilot for Windows 11

Although Copilot has been available on Windows 11 for some time, starting with version 24H2, Microsoft will be making some changes. For example, in this new version, Copilot becomes a traditional app that you can pin or unpin from the Taskbar. You can also resize and move it around like any other app. (This app is also available on Windows 11 23H2.)

The app is actually the web version of Copilot installed as a web app on Windows 11. It works as before, but the difference is that it can no longer change system settings. Also, the interface has been updated with a new design similar to the OpenAI ChatGPT experience on the web.

Copilot is also coming to the Settings app. However, you won’t get a button to summon the chatbot. Instead, you will find “Copilot suggestions” within the settings. For example, if you’re in the System section, you may see a collection of suggestions to fix audio problems, turn on the energy-saving feature, or cast your screen to a wireless display.

On File Explorer, Microsoft plans to bring Copilot to help you get things done in fewer steps. For example, if you are in the Gallery tab, you can hover over a picture and clicking the Copilot button will bring up different actions you can apply to the image file.

Furthermore, Copilot is also coming to Notifications. When you receive a toast notification, Copilot will appear with a suggestion to perform in that specific case. For example, if you’re getting an email with a PowerPoint attachment, the chatbot may suggest summarising the content.

On Windows 11, this also means that you will be able to share your screen with Copilot to get assistance with anything you’re doing.

It’s also important to point out that developers will now start integrating AI into their apps, meaning that starting with the next version of the operating system, we will start seeing more apps with AI capabilities.

The Copilot changes were first unveiled during the last Surface launch event, but since then, the company hasn’t shared any more details about the Copilot suggestions.

Wrapping things up

The Windows 11 2024 Update is available now with many new features and improvements but without any of the AI features. Microsoft is expected to start testing the AI features (such as Windows Recall with the new security improvements) sometime in October with Copilot+ PCs through the Windows Insider Program with feature arriving in November 2024.

Copilot+ PCs are a new category of computers with AI capabilities. The first wave of devices features Qualcomm Snapdragon X series processors with integrated NPUs to handle virtually any AI feature and provide a true all-day battery life. These devices come with a minimum of 16GB of memory and 256GB of storage. Intel and AMD are also expected to ship processors with NPUs to handle these new AI experiences.

Some of these new Copilot+ PCs include the Surface Pro (11th Edition), Surface Laptop (7th Edition), Acer Swift 14 AI, HP Omnibook X, HP EliteBook Ultra G1q, Lenovo Yoga Slim 7x, Samsung Galaxy Book4 Edge series, ASUS Vivobook S 15, and Dell laptops like the XPS 13, Inspiron 14, Inspiron 14 Plus, Latitude 5455 and Latitude 7455.

Update October 8, 2024: This page has been updated to ensure accuracy and include the latest information.